9/8/2018 


Special Distributions | Bernoulli Distribution | Geometric Distribution | Binomial Distribution [ Pascal Distribution | Poisson Distribution 


3.1.5 Special Distributions 

As it turns out, there are some specific distributions that are used over and over in practice, thus they have been given special names. There is a 
random experiment behind each of these distributions. Since these random experiments model a lot of real life phenomenon, these special 
distributions are used frequently in different applications. That's why they have been given a name and we devote a section to study them. We 
will provide PMFs for all of these special random variables, but rather than trying to memorize the PMF, you should understand the random 
experiment behind each of them. If you understand the random experiments, you can simply derive the PMFs when you need them. Although it 
might seem that there are a lot of formulas in this section, there are in fact very few new concepts. Do not get intimidated by the large number 
of formulas, look at each distribution as a practice problem on discrete random variables. 


Bernoulli Distribution 

What is the simplest discrete random variable (i.e., simplest PMF) that you can imagine? My answer to this question is a PMF that is nonzero 
at only one point. For example, if you define 



1 for x — 1 

0 otherwise 


then X is a discrete random variable that can only take one value, i.e., X = 1 with a probability of one. But this is not a very interesting 
distribution because it is not actually random. Then, you might ask what is the next simplest discrete distribution. And my answer to that is the 
Bernoulli distribution. A Bernoulli random variable is a random variable that can only take two possible values, usually 0 and 1 . This random 
variable models random experiments that have two possible outcomes, sometimes referred to as "success" and "failure." Here are some 
examples: 


• You take a pass-fail exam. You either pass (resulting in X = 1) or fail (resulting in X = 0). 

• You toss a coin. The outcome is ether heads or tails. 

• A child is born. The gender is either male or female. 


Formally, the Bernoulli distribution is defined as follows: 


Definition 3.4 

A random variable X is said to be a Bernoulli random variable with parameter p, shown as X ~ Bernoulli (p), if its PMF is given 
by 

f p 

for x = 1 

Px( x ) = < 1 -p 

for x = 0 

lo 

otherwise 

where 0 < p < 1. 



Figure 3.2 shows the PMF of a Bernoulli (jp ) random variable. 
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X rv Bernoulliip) 



0 1 


Fig.3.2 - PMF of a Bemoulli[p ) random variable. 

A Bernoulli random variable is associated with a certain event A. If event A occurs (for example, if you pass the test), then X = 1; 
otherwise X = 0. For this reason the Bernoulli random variable, is also called the indicator random variable. In particular, the indicator 
random variable I a for an event A is defined by 



1 if the event A occurs 

0 otherwise 


The indicator random variable for an event A has Bernoulli distribution with parameter p = P(A), so we can write 


I A ~ Bernoulli (P(A)Y 


Geometric Distribution 

The random experiment behind the geometric distribution is as follows. Suppose that I have a coin with P(H) = p. I toss the coin until I 
observe the first heads. We define X as the total number of coin tosses in this experiment. Then X is said to have geometric distribution with 
parameter p. In other words, you can think of this experiment as repeating independent Bernoulli trials until observing the first success. This is 

exactlv the same distribution that we saw in Example 3.4. The range of X here is Rx = {1,2,3, ... }. In Example 3.4, we obtained 

P x {k) = P{X=k ) = (1 -p) k ~ l p, for k = 1,2,3,... 

We usually define q = 1 — p, so we can write Px (fc) = pq k ~^ , for k — 1, 2, 3, .... To say that a random variable has geometric 
distribution with parameter p, we write X ~ Geometric(p ). More formally, we have the following definition: 
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Definition 3.5 

A random variable X is said to be a geometric random variable with parameter p, shown as X ~ Geometric(jp), if its PMF is 
given by 


Px(k) 


( p( 1 ~ p) k 1 

10 


for k = 1,2, 3,... 
otherwise 


where 0 < p < 1 . 


Figure 3.3 shows the PMF of a Geometric (0.3) random variable. 


X rv Geometric(p = ( 



« - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - x - m- 

0123456789 10 11 12 


Fig.3.3 - PMF of a Geometric^ 0.3) random variable. 
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We should note that some books define geometric random variables slightly differently. They define the geometric random variable X as the 
total number of failures before observing the first success. By this definition the range of X is Rx = {0, 1,2,...} and the PMF is given 
by 


P x (k) = | p ( 1 _p ) fc for k = 0,1,2,3,... 

10 otherwise 

In this book, whenever we write X ~ Geometric(jp) , we always mean X as the total number of trials as defined in Definition 3.5. Note 
that as long as you are consistent in your analysis, it does not matter which definition you use. That is why we emphasize that you should 
understand how to derive PMFs for these random variables rather than memorizing them. 

Binomial Distribution 

The random experiment behind the binomial distribution is as follows. Suppose that I have a coin with = p. I toss the coin n times 

and define X to be the total number of heads that I observe. Then X is binomial with parameter n and p, and we write 
X ~ Binomial(n , p) . The range of X in this case is Rx = {0,1,2,..., n}. As we have seen in Section 2.1.3, the PMF of X in 
this case is given by binomial formula 

Px(k) = \V\p k {l-p) n ~ k , for k = 0,1, 2,... ,n. 


We have the following definition: 


Definition 3.6 

A random variable X is said to be a binomial random variable with parameters n and p, shown as X ~ Binomial[n , p ), if its PMF 
is given by 


ft (*) - {O * 1 - 


for k = 0,1,2, • • •, n 
otherwise 


where 0 < p < 1 . 


Figures 3.4 and 3.5 show the Binomial(n,p) PMF for n = 10, p = 0.3 and n = 20, p = 0.6 respectively. 
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X ~ Binomial(n = 10 ,p 



0 1 2 3 4 5 6 

Fig.3.4 - PMF of a Binomial^ 10, 0.3) random variable. 
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X ~ Binomial(n = 20, p 



Binomial random variable as a sum of Bernoulli random variables 


Here is a useful way of thinking about a binomial random variable. Note that a Binomial(n, p) random variable can be obtained by Tl 
independent coin tosses. If we think of each coin toss as a Bernoulli (jp) random variable, the Binomial(n , p) random variable is a 
sum of n independent Bemoulli(p) random variables. This is stated more precisely in the following lemma. 


Lemma 3.1 


If X \, X 2 , . . . , X n are independent Bernoulli (p) random variables, then the random variable X defined by 
X = X\ + X 2 ~h- • • ~hX n has a Binomial(n,p) distribution. 

To generate a random variable X ^ Binomial[n , p), we can toss a coin n times and count the number of heads. Counting the number of 

heads is exactly the same as finding X\ + X 2 + . . . +X n , where each X{ is equal to one if the corresponding coin toss results in heads and 
zero otherwise. This interpretation of binomial random variables is sometimes veiy helpful. Let's look at an example. 


Example 3.7 

Let X Binomial(n , p) and Y ~ Binomial(m , p) be two independent random variables. Define a new random variable as 

Z = X + Y. Find the PMF of Z. 

• Solution 

o Since X Binomial{n^p), we can think of X as the number of heads in n independent coin tosses, i.e., we can write 

X = X\ + X 2 ~K .. 

where the X{' s are independent Bernoulli (p) random variables. Similarly, since Y ~ Binomial (m, p), we can think of 
Y as the number of heads in m independent coin tosses, i.e., we can write 

Y = Y 1 + Y 2 +...+Y m , 
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where the Yj's are independent Bernoulli{jp ) random variables. Thus, the random variable Z = X -f- Y will be the total 
number of heads in 72 + 772 independent coin tosses: 

z = x + r = x 1 + x 2 +... +x n + y 1 + r 2 +... 

where the Xf s and Yj's are independent Bernoulli(p) random variables. Thus, by Lemma 3.1, Z is a binomial random 
variable with parameters 772 + n andp, i.e., Binomial(m + 72, p). Therefore, the PMF of Z is 



( m + n )p k (l-p) m+n - k 

0 


for k = 0,1,2,3,..., m + n 
otherwise 


The above solution is elegant and simple, but we may also want to directly obtain the PMF of Z using probability rules. Here is 
another method to solve Example 3.7. First, we note that R z = {0,1,2 ,...,771 + 72}. For k £ Rz, we can write 

P z (k ) = P(Z = k)= P(X + Y = k). 


We will find P[X -\-Y = k) by using conditioning and the law of total probability. In particular, we can write 

P z {k) = P{X + Y = k) 

= +Y = k\X = i)P(X = i) (law of total probability) 

= YZ=0 P(Y = k - i\X = i)P(X = i) 


= E"=o P(Y = k- i)P{X = i) 

= e:=o -p) ra - fc+i (>hi -p) 

= eiu ( fc E)O fc (i-pr + ”- fc 

=/( i_ pr+ n-, ELo ( ^ )(:) 

_ ( m + n )p fc (l — p) m+n_fc 


n—i 


(since X and Y are independent) 
(since X and Y are binomial) 


(by Example 2.8 (part 3)). 


Thus, we have proved Z ~ Binomial(m + n, p) by directly finding the PMF of Z . 


Negative Binomial (Pascal) Distribution 

The negative binomial or Pascal distribution is a generalization of the geometric distribution. It relates to the random experiment of repeated 
independent trials until observing m successes. Again, different authors define the Pascal distribution slightly differently, and as we mentioned 
before if you understand one of them you can easily derive the other ones. Here is how we define the Pascal distribution in this book. Suppose 
that I have a coin with P[H) = p. I toss the coin until I observe Vfl heads, where 772 £ N. We define X as the total number of coin tosses 
in this experiment. Then X is said to have Pascal distribution with parameter Vfl and p. We write X ~ Pascal^m, p) . Note that 
Pascal(l,p) = Geometric(p) . Note that by our definition the range of X is given by Rx={ni, Vfl + 1, Vfl + 2, Vfl + 3, • • •}. 

Let us derive the PMF of a Pascal{rn , p) random variable X. Suppose that I toss the coin until I observe Vfl heads, and X is defined as 
the total number of coin tosses in this experiment. To find the probability of the event A = { X = &}, we argue as follows. By definition, 
event A can be written as A = B n c, where 

• B is the event that we observe 772—1 heads (successes) in the first k — 1 trials, and 

• C is the event that we observe a heads in the kth trial. 


Note that B and C are independent events because they are related to different independent trials (coin tosses). Thus we can write 


P(A) = P(BnC) = P(B)P(C). 


Now, we have P(C ) = p. Note also that P[B ) is the probability that I observe observe 772 — 1 heads in the k — 1 coin tosses. This 
probability is given by the binomial formula, in particular 


P(B) = ( k \)p™-\l = [ k 1 V-hl-p) 

\m — 1/ \772 — 1/ 


k—m 


Thus, we obtain 


P(A) = P(BHC) = P(B)P(C) 


fk- 1 

\TO — 1 


P m (l-P) k ~ m . 


To summarize, we have the following definition for the Pascal random variable 
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Definition 3.7 

A random variable X is said to be a Pascal random variable with parameters m and p, shown as X ~ Pascal{m , p) , if its PMF is 
given by 



im-i)p m (l~P) k m for k = m,m + l,m + 2,m + 3,... 

0 otherwise 


where 0 < p < 1. 


Figure 3.6 shows the PMF of a Pascal(m,p ) random variable with m = 3 andp = 0.5. 


X ~ NegativeBinomial (m = 
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Hypergeometric Distribution 


Here is the random experiment behind the hypergeometric distribution. You have a bag that contains b blue marbles and V red marbles. You 
choose k < b + r marbles at random (without replacement). Let X be the number of blue marbles in your sample. By this definition, we 
have X < min(ft, b). Also, the number of red marbles in your sample must be less than or equal to r, so we conclude 
X > max(0, k — r). Therefore, the range of X is given by 

Rx = {max(0, k — r), max(0, k — r) + 1, max(0, k — r) + 2,..., min(&, 6)}. 


To find Px ( X ), note that the total number of ways to choose k marbles from b -f- r marbles is ( & ^ r ) 
blue marbles and k — X red marbles is Q (k-x) * Thus, we have 


The total number of ways to choose x 



The following definition summarizes the discussion above. 


for x e Rx • 


Definition 3.8 

A random variable X is said to be a Hypergeometric random variable with parameters 6, T and k, shown as 
X rsj Hyper geometric(b, r, k), if its range is 

Rx = {max(0,& — r),max(0,ft — r) + l,max(0,fc — r) + 2,..., min(&, b)}, and its PMF is given by 

^*j ^ for x € Rx 

C r) 

0 otherwise 


Px(x) = { 


Again, there is no point to memorizing the PMF. All you need to know is how to solve problems that can be formulated as a hypergeometric 
random variable. 


Poisson Distribution 

The Poisson distribution is one of the most widely used probability distributions. It is usually used in scenarios where we are counting the 
occurrences of certain events in an interval of time or space. In practice, it is often an approximation of a real-life random variable. Here is an 
example of a scenario where a Poisson random variable might be used. Suppose that we are counting the number of customers who visit a 
certain store from 1 pm to 2 pm. Based on data from previous days, we know that on average A = 15 customers visit the store. Of course, 
there will be more customers some days and fewer on others. Here, we may model the random variable X showing the number customers as a 
Poisson random variable with parameter A = 15. Let us introduce the Poisson PMF first, and then we will talk about more examples and 
interpretations of this distribution. 


Definition 3.9 

A random variable X is said to be a Poisson random variable with parameter A, shown as X ~ Poisson(X) , if its range is 
Rx = { 0 , 1,2,3,...}, and its PMF is given by 


Px(k) 


for k £ Rx 
0 otherwise 


Before going any 

E* e , Rx Px(k ) 


further, let's check that this is a valid PMF. First, we note that Px (k) > 0 for all k. Next, we need to check 
= 1. To do that, let us first remember the Taylor series for e x , e x = p\' Now we can write 



e- x X k 

Z^k=0 k\ 
e 2^k =o k\ 

e~ x e x (by Taylor series for e x ) 

1 . 
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Figures 3.7, 3.8, and 3.9 show the Poisson(X) PMF for A = 1, A = 5, and A = 10 respectively. 


X Poisson( A = ] 



0123456789 10 11 12 


Fig.3.7 - PMF of a Poisson(l ) random variable. 
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X rv Poisson( A = l 



0123456789 10 11 12 


Fig.3.8 - PMF of a Poisson( 5) random variable. 
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X Poisson( A = 1 



Now let's look at an example. 


Example 3.8 

The number of emails that I get in a weekday can be modeled by a Poisson distribution with an average of 0.2 emails per minute. 

1. What is the probability that I get no emails in an interval of length 5 minutes? 

2. What is the probability that I get more than 3 emails in an interval of length 10 minutes? 


Solution 


1. Let X be the number of emails that I get in the 5-minute interval. Then, by the assumption X is a Poisson random variable 
with parameter A = 5(0.2) = 1, 


P(x = 0) = Px{ 0) = 


r A A° 

0 ! 


• 1 


= - « 0.3679 

e 


2. Let Y be the number of emails that I get in the 10 -minute interval. Then by the assumption Y is a Poisson random variable 
with parameter A = 10(0.2) = 2, 

P{Y > 3) = 1 - P(Y < 3) 

= 1 - (JV( o) + JY(1) + Py( 2) + Py{ 3)) 

A e“ A A e a A 2 e~ A A 3 


= l-e~ A - 
= l-e~ 2 - 


1! 

2e" 2 


2! 

4e -2 


3! 


8e 


1 2 

= l-e- 2 (1 + 2 + 2 +|) 

= 1 ~ 3 ~ °' 1429 


6 
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Poisson as an approximation for binomial 

The Poisson distribution can be viewed as the limit of binomial distribution. Suppose X ~ Binomial(n , p ) where Tl is very large andp 
is very small. In particular, assume that A = up is a positive constant. We show that the PMF of X can be approximated by the PMF of a 
Poisson(X) random variable. The importance of this is that Poisson PMF is much easier to compute than the binomial. Let us state this as a 
theorem. 


Theorem 3.1 


Let X ~ Binomial (n, P = where A > 0 is fixed. Then for any k E {0, 1, 2, . . . }, we have 


lim Px{k ) 

n—>oo 




Proof 
We have 


L n—oo 


limn^oo Px(k) = lim 

= X k lim n _ i . O0 


©(T)VTT fc 


n! 




n—k 


\k 

— yv . lim 

k\ 


k\(n—k)\ 

n(n— l)(n—2)...(n— k+1) 


■n—>■ oo 


rr 



V- 

L - A T 


7i_ Ar*' 

i 


V 

n / 


V n) 

) 


Note that for a fixed k, we have 


linin^oo 

li m n->oo 

linin^oo 


n(n— l)(n—2)...(n—fe+1) 
n k 



1 


Thus, we conclude 


lim Px(k ) 

n—> oo 




<— previous 
next —> 
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